{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "\n",
    "\n",
    "# 数据准备\n",
    "## 工具包 & 数据导入\n",
    "### 工具包导入"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "import lightgbm as lgb\n",
    "import gc\n",
    "%matplotlib inline "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 数据导入\n",
    "- 训练集测试集的导入"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "path = '/data/Data_JieZhang/Inscart/'\n",
    "aisles = pd.read_csv(path + 'aisles.csv')\n",
    "departments = pd.read_csv(path + 'departments.csv')\n",
    "products = pd.read_csv(path + 'products.csv')\n",
    "orders = pd.read_csv(path + 'orders.csv')\n",
    "order_products__train = pd.read_csv(path + 'order_products__train.csv')\n",
    "order_products__prior = pd.read_csv(path + 'order_products__prior.csv')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 特征工程\n",
    "\n",
    "目前很多比赛都还是特征为王的比赛，针对许多传统的比赛，因为存在很多封装好的工具包,例如XGBoost,LightGBM等工具包,所以很多做数据分析建模或者数据竞赛的人的任务往往就会相对简单，这个时候因为模型变化不大,那么我们的任务往往就会演变为两大模块,特征工程+集成，而特征工程又很难找到非常好的书籍进行系统的学习,所以很多时候网上也都将特征工程称之为艺术。\n",
    "\n",
    "闲话不多说,下面我们还是以推荐问题为例,介绍基本的推荐算法的特征工程。\n",
    "\n",
    "我们将该赛题的特征工程粗分为下面几个模块。\n",
    "\n",
    "1. 用户特征;\n",
    "2. 产品特征;\n",
    "3. 用户产品特征;\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 用户特征 "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>order_id</th>\n",
       "      <th>user_id</th>\n",
       "      <th>eval_set</th>\n",
       "      <th>order_number</th>\n",
       "      <th>order_dow</th>\n",
       "      <th>order_hour_of_day</th>\n",
       "      <th>days_since_prior_order</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2539329</td>\n",
       "      <td>1</td>\n",
       "      <td>prior</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>8</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2398795</td>\n",
       "      <td>1</td>\n",
       "      <td>prior</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>7</td>\n",
       "      <td>15.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>473747</td>\n",
       "      <td>1</td>\n",
       "      <td>prior</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>12</td>\n",
       "      <td>21.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2254736</td>\n",
       "      <td>1</td>\n",
       "      <td>prior</td>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>7</td>\n",
       "      <td>29.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>431534</td>\n",
       "      <td>1</td>\n",
       "      <td>prior</td>\n",
       "      <td>5</td>\n",
       "      <td>4</td>\n",
       "      <td>15</td>\n",
       "      <td>28.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   order_id  user_id eval_set  order_number  order_dow  order_hour_of_day  \\\n",
       "0   2539329        1    prior             1          2                  8   \n",
       "1   2398795        1    prior             2          3                  7   \n",
       "2    473747        1    prior             3          3                 12   \n",
       "3   2254736        1    prior             4          4                  7   \n",
       "4    431534        1    prior             5          4                 15   \n",
       "\n",
       "   days_since_prior_order  \n",
       "0                     NaN  \n",
       "1                    15.0  \n",
       "2                    21.0  \n",
       "3                    29.0  \n",
       "4                    28.0  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "orders.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 用户购物的总次数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "user_fea = pd.DataFrame()\n",
    "user_fea['user_id'] = orders['user_id'].unique()\n",
    "user_fea = user_fea[['user_id']].sort_values('user_id')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "user_fea['user_orderid_count'] = orders.groupby('user_id')['order_id'].count().values "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 用户购物频繁度\n",
    "\n",
    "1. 如果用户平均相隔天数少,往往说明用户是爱频繁购物的，那么他很有可能会不断地去消费某些产品;\n",
    "2. 如果用户距离上次的购物最大天数都很小,那么往往说明客户是非常爱购物的,和均值特征是类似的一种.\n",
    "3. 如果用户距离上次购物的时间的方差较小,说明用户购物有较强的周期性.\n",
    "4. 同样的,用户距离上一次购买产品的众数也是此类特征之一."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "user_fea['user_days_since_prior_order_mean'] = orders.groupby('user_id')['days_since_prior_order'].mean().values\n",
    "user_fea['user_days_since_prior_order_max'] = orders.groupby('user_id')['days_since_prior_order'].max().values\n",
    "user_fea['user_days_since_prior_order_std'] = orders.groupby('user_id')['days_since_prior_order'].std().values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "user_fea['user_days_since_prior_order_mode'] = orders.groupby('user_id')['days_since_prior_order'].apply(lambda x: x.mode()[0]).values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 用户时间喜好\n",
    "- 不同用户购物的时间往往不同,很多用户喜欢深夜购物,有的用户则往往喜欢周五购物,这些不同的时间点往往也是一个非常重要的选择,比如有些用户喜欢周五买香蕉什么的生活习惯,可以对预测带来非常大的帮助。有人则喜欢深夜购物等等。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "user_fea['user_order_dow_mode']         = orders.groupby('user_id')['order_dow'].apply(lambda x: x.mode()[0]).values\n",
    "user_fea['user_order_hour_of_day_mode'] = orders.groupby('user_id')['order_hour_of_day'].apply(lambda x: x.mode()[0]).values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 补充上一个二阶的用户时间喜好特征,用来表示用户喜欢在每周晚上几点开始购物."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "orders['dow_hour'] = orders['order_dow'].values * 25 + orders['order_hour_of_day'].values\n",
    "user_fea['user_dow_hour_mode'] = orders.groupby('user_id')['dow_hour'].apply(lambda x: x.mode()[0]).values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 用户商品喜好特征\n",
    "- 注意此处我们以priors的数据集进行商品特征的提取,**可能很多人会认为为什么不结合train一起提取特征**,这样数据量更大,这主要是考虑到test中并无类似的特征,而且test中的用户又未出现在train中,所以这样提取的特征是有些问题的,因为训练集中的特征来源于priors & train,但是test却只能来源于prior,所以可能造成偏差。\n",
    "\n",
    "此处我们用如下的几类特征来表示用户商品的特征.\n",
    "1. 用户购买不同产品的数量\n",
    "2. 用户购买最多的三种产品,如果用户只购买了两类产品,那么最后一种产品我们用-1来表示.\n",
    "3. 用户购买所有产品的数量\n",
    "4. 用户平均每一单购买的产品数量"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "order_products__prior_ = order_products__prior.merge(orders, on='order_id', how = 'left')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "user_fea['user_product_nunique'] = order_products__prior_.groupby('user_id')['product_id'].nunique().sort_index().values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "def mode_N(x,i):\n",
    "    m = x.value_counts().index\n",
    "    if len(m) > i: \n",
    "        return m[i] \n",
    "    return -1\n",
    "user_fea['user_product_mode'] = order_products__prior_.groupby('user_id')['product_id'].apply(lambda x: x.mode()[0]).sort_index().values\n",
    "user_fea['user_product_mode_1'] = order_products__prior_.groupby('user_id')['product_id'].apply(lambda x: mode_N(x,1)).sort_index().values \n",
    "user_fea['user_product_mode_2'] = order_products__prior_.groupby('user_id')['product_id'].apply(lambda x: mode_N(x,2)).sort_index().values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "user_fea['user_product_count'] = order_products__prior_.groupby('user_id')['product_id'].count().sort_index().values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "user_fea['user_product_orderid_ratio'] = user_fea['user_product_count'] / user_fea['user_orderid_count']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 用户购物疯狂性特征\n",
    "1. 用户订单中的商品最多数\n",
    "2. 用户每次平均购买的商品量\n",
    "3. 用户购买商品的1/4,3/4位数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "tmp = order_products__prior_.groupby(['user_id','order_id'])['add_to_cart_order'].max().reset_index()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "user_fea['user_add_to_cart_order_max'] = tmp.groupby('user_id')['add_to_cart_order'].max().sort_index().values\n",
    "user_fea['user_add_to_cart_order_mean'] = tmp.groupby('user_id')['add_to_cart_order'].mean().sort_index().values\n",
    "user_fea['user_add_to_cart_order_quantile_25'] = tmp.groupby('user_id')['add_to_cart_order'].quantile().sort_index().values\n",
    "user_fea['user_add_to_cart_order_quantile_75'] = tmp.groupby('user_id')['add_to_cart_order'].quantile(0.75).sort_index().values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "61"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "del tmp\n",
    "gc.collect()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 用户购买商品的重构率\n",
    "\n",
    "我们用用户购买的所有商品中的重新购买的和以及均值来表示"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "user_fea['user_reordered_sum'] = order_products__prior_.groupby('user_id')['reordered'].sum().sort_index().values\n",
    "user_fea['user_reordered_mean'] = order_products__prior_.groupby('user_id')['reordered'].mean().sort_index().values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 基于产品的特征\n",
    "\n",
    "上面我们对用户特征进行了简单的提取,分别针对用户的购买时间偏好,用户对商品的喜好,重构的情况等等出发进行特征的构建。\n",
    "\n",
    "下面我们从商品的特征出发进行商品特征的构建。虽然下面的特征都可以认为是产品的热度&受欢迎度特征等,但我们还是尽可能将其分开解释。\n",
    "\n",
    "### 产品的热度\n",
    "1. 产品被买多少次\n",
    "2. 产品被多少不同用户所购买\n",
    "3. 产品被添加到篮子中的顺序的均值 & 标准差"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "product_fea = pd.DataFrame()\n",
    "product_fea['product_id'] = order_products__prior_['product_id'].unique() \n",
    "product_fea= product_fea.sort_values('product_id')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "product_fea['product_count'] = order_products__prior_.groupby('product_id')['user_id'].count().sort_index().values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "product_fea['product_order_nunqiue'] = order_products__prior_.groupby('product_id')['order_id'].nunique().sort_index().values\n",
    "product_fea['product_user_nunqiue']  = order_products__prior_.groupby('product_id')['user_id'].nunique().sort_index().values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "product_fea['product_add_to_cart_order_mean'] = order_products__prior_.groupby('product_id')['add_to_cart_order'].mean().sort_index().values \n",
    "product_fea['product_add_to_cart_order_std']  = order_products__prior_.groupby('product_id')['add_to_cart_order'].std().sort_index().values "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 产品受欢迎的时间段\n",
    "1. 产品被购买最多的两天\n",
    "2. 产品被购买最多的两个小时段\n",
    "3. 产品被购买距离上次购买最多的两个时间段\n",
    "4. 产品距离上次被购买的均值以及均方差"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "product_fea['product_dow_mode'] = order_products__prior_.groupby('product_id')['order_dow'].apply(lambda x: x.mode()[0]).sort_index().values\n",
    "product_fea['product_dow_mode_1'] = order_products__prior_.groupby('product_id')['order_dow'].apply(lambda x: mode_N(x,1)).sort_index().values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "product_fea['product_hour_of_day_mode'] = order_products__prior_.groupby('product_id')['order_hour_of_day'].apply(lambda x: x.mode()[0]).sort_index().values\n",
    "product_fea['product_hour_of_day_mode_1'] = order_products__prior_.groupby('product_id')['order_hour_of_day'].apply(lambda x: mode_N(x,1)).sort_index().values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "product_fea['product_days_since_prior_order_mode'] = order_products__prior_.groupby('product_id')['days_since_prior_order'].apply(lambda x: mode_N(x,0)).sort_index().values\n",
    "product_fea['product_days_since_prior_order_mode_1'] = order_products__prior_.groupby('product_id')['days_since_prior_order'].apply(lambda x: mode_N(x,1)).sort_index().values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "product_fea['product_dow_hour_mode'] = order_products__prior_.groupby('product_id')['dow_hour'].apply(lambda x: mode_N(x,0)).sort_index().values\n",
    "product_fea['product_dow_hour_mode_1'] = order_products__prior_.groupby('product_id')['dow_hour'].apply(lambda x: mode_N(x,1)).sort_index().values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "product_fea['product_days_since_prior_order_mean'] = order_products__prior_.groupby('product_id')['days_since_prior_order'].mean().sort_index().values \n",
    "product_fea['product_days_since_prior_order_std'] = order_products__prior_.groupby('product_id')['days_since_prior_order'].std().sort_index().values "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 产品的被重购率\n",
    "\n",
    "产品的重构率,标准差以及和。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "product_fea['product_reordered_mean'] = order_products__prior_.groupby('product_id')['reordered'].mean().sort_index().values \n",
    "product_fea['product_reordered_std'] = order_products__prior_.groupby('product_id')['reordered'].std().sort_index().values \n",
    "product_fea['product_reordered_sum'] = order_products__prior_.groupby('product_id')['reordered'].sum().sort_index().values "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 基于用户 + 产品的交叉特征"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "order_products__prior_['user_product'] = order_products__prior_['user_id'].values * 10**5 + order_products__prior_['product_id'].values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "userXproduct_fea = pd.DataFrame()\n",
    "userXproduct_fea['user_product'] = order_products__prior_['user_product'].unique() \n",
    "userXproduct_fea = userXproduct_fea[['user_product']].sort_values('user_product')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**此处的用户和产品的交叉特征,我们就用下面几个简单特征进行表述**:\n",
    "1. 同一用户对某一个产品的重构率;\n",
    "2. 同一用户对某一个产品的加入篮子的顺序的和以及均值;\n",
    "3. 同一用户购买某一个产品的次数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "userXproduct_fea['user_product_reordered_sum'] = order_products__prior_.groupby('user_product')['reordered'].sum().sort_index().values "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "userXproduct_fea['user_product_add_to_cart_order_sum']   = order_products__prior_.groupby('user_product')['add_to_cart_order'].sum().sort_index().values \n",
    "userXproduct_fea['user_product_add_to_cart_order_mean']   = order_products__prior_.groupby('user_product')['add_to_cart_order'].mean().sort_index().values "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "userXproduct_fea['user_product_order_nunique']   = order_products__prior_.groupby('user_product')['order_id'].nunique().sort_index().values "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "userXproduct_fea['user_product_last_order_num'] = order_products__prior_.groupby('user_product')['order_number'].max().sort_index().values "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_product</th>\n",
       "      <th>user_product_reordered_sum</th>\n",
       "      <th>user_product_add_to_cart_order_sum</th>\n",
       "      <th>user_product_add_to_cart_order_mean</th>\n",
       "      <th>user_product_order_nunique</th>\n",
       "      <th>user_product_last_order_num</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>3105781</th>\n",
       "      <td>100196</td>\n",
       "      <td>9</td>\n",
       "      <td>14</td>\n",
       "      <td>1.400000</td>\n",
       "      <td>10</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3105783</th>\n",
       "      <td>110258</td>\n",
       "      <td>8</td>\n",
       "      <td>30</td>\n",
       "      <td>3.333333</td>\n",
       "      <td>9</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3105785</th>\n",
       "      <td>110326</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3105782</th>\n",
       "      <td>112427</td>\n",
       "      <td>9</td>\n",
       "      <td>33</td>\n",
       "      <td>3.300000</td>\n",
       "      <td>10</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3767345</th>\n",
       "      <td>113032</td>\n",
       "      <td>2</td>\n",
       "      <td>19</td>\n",
       "      <td>6.333333</td>\n",
       "      <td>3</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         user_product  user_product_reordered_sum  \\\n",
       "3105781        100196                           9   \n",
       "3105783        110258                           8   \n",
       "3105785        110326                           0   \n",
       "3105782        112427                           9   \n",
       "3767345        113032                           2   \n",
       "\n",
       "         user_product_add_to_cart_order_sum  \\\n",
       "3105781                                  14   \n",
       "3105783                                  30   \n",
       "3105785                                   5   \n",
       "3105782                                  33   \n",
       "3767345                                  19   \n",
       "\n",
       "         user_product_add_to_cart_order_mean  user_product_order_nunique  \\\n",
       "3105781                             1.400000                          10   \n",
       "3105783                             3.333333                           9   \n",
       "3105785                             5.000000                           1   \n",
       "3105782                             3.300000                          10   \n",
       "3767345                             6.333333                           3   \n",
       "\n",
       "         user_product_last_order_num  \n",
       "3105781                           10  \n",
       "3105783                           10  \n",
       "3105785                            5  \n",
       "3105782                           10  \n",
       "3767345                           10  "
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "userXproduct_fea.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 构建训练集 & 测试集\n",
    "具体的构建原理此处就不再阐述,具体的可以参见前一篇文章的内容:Instacart Market Basket Analysis：数据探索分析&Baseline(0.2428)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "orders_prior_data = orders.loc[orders.eval_set == 'prior']\n",
    "orders_train_data = orders.loc[orders.eval_set == 'train'] \n",
    "orders_test_data  = orders.loc[orders.eval_set == 'test' ] \n",
    "\n",
    "priors = order_products__prior.merge(orders_prior_data, on =['order_id'], how='left')\n",
    "trains = order_products__train.merge(orders_train_data, on =['order_id'], how='left')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "user_product = order_products__prior_[['user_id','product_id']].copy()\n",
    "user_product['user_X_product'] = user_product['user_id'].values* 10**5  + user_product['product_id'].values\n",
    "train_user_X_product = trains['user_id'].values* 10**5 + trains['product_id'].values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "user_product = user_product.drop_duplicates(subset=['user_X_product'], keep = 'last') "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "test_user  = orders_test_data['user_id']\n",
    "train_user = orders_train_data['user_id']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/conda/envs/zjpy36/lib/python3.6/site-packages/pandas/core/indexing.py:194: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame\n",
      "\n",
      "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy\n",
      "  self._setitem_with_indexer(indexer, value)\n",
      "/opt/conda/envs/zjpy36/lib/python3.6/site-packages/ipykernel_launcher.py:3: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame\n",
      "\n",
      "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy\n",
      "  This is separate from the ipykernel package so we can avoid doing imports until\n"
     ]
    }
   ],
   "source": [
    "user_product['label'] = 0\n",
    "train_data = user_product.loc[user_product.user_id.isin(train_user)]\n",
    "train_data.loc[train_data.user_X_product.isin(train_user_X_product), 'label'] = 1 "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "828824"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_data['label'].sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.09780025419305857"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_data['label'].mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_data = train_data.merge(orders_train_data,on ='user_id', how='left')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [],
   "source": [
    "test_data  = user_product.loc[user_product.user_id.isin(test_user)]\n",
    "test_data = test_data.merge(orders_test_data,on ='user_id', how='left')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 特征拼接"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- orders的数据合并"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_data = train_data.merge(user_fea, on='user_id', how='left')\n",
    "train_data = train_data.merge(product_fea, on='product_id', how='left')\n",
    "train_data = train_data.merge(userXproduct_fea, left_on='user_X_product', right_on='user_product', how='left')\n",
    "train_data = train_data.merge(products, on='product_id', how= 'left')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_id</th>\n",
       "      <th>product_id</th>\n",
       "      <th>user_X_product</th>\n",
       "      <th>label</th>\n",
       "      <th>order_id</th>\n",
       "      <th>eval_set</th>\n",
       "      <th>order_number</th>\n",
       "      <th>order_dow</th>\n",
       "      <th>order_hour_of_day</th>\n",
       "      <th>days_since_prior_order</th>\n",
       "      <th>dow_hour</th>\n",
       "      <th>user_orderid_count</th>\n",
       "      <th>user_days_since_prior_order_mean</th>\n",
       "      <th>user_days_since_prior_order_max</th>\n",
       "      <th>user_days_since_prior_order_std</th>\n",
       "      <th>user_days_since_prior_order_mode</th>\n",
       "      <th>user_order_dow_mode</th>\n",
       "      <th>user_order_hour_of_day_mode</th>\n",
       "      <th>user_dow_hour_mode</th>\n",
       "      <th>user_product_nunique</th>\n",
       "      <th>user_product_mode</th>\n",
       "      <th>user_product_mode_1</th>\n",
       "      <th>user_product_mode_2</th>\n",
       "      <th>user_product_count</th>\n",
       "      <th>user_product_orderid_ratio</th>\n",
       "      <th>...</th>\n",
       "      <th>product_user_nunqiue</th>\n",
       "      <th>product_add_to_cart_order_mean</th>\n",
       "      <th>product_add_to_cart_order_std</th>\n",
       "      <th>product_dow_mode</th>\n",
       "      <th>product_dow_mode_1</th>\n",
       "      <th>product_hour_of_day_mode</th>\n",
       "      <th>product_hour_of_day_mode_1</th>\n",
       "      <th>product_days_since_prior_order_mode</th>\n",
       "      <th>product_days_since_prior_order_mode_1</th>\n",
       "      <th>product_dow_hour_mode</th>\n",
       "      <th>product_dow_hour_mode_1</th>\n",
       "      <th>product_days_since_prior_order_mean</th>\n",
       "      <th>product_days_since_prior_order_std</th>\n",
       "      <th>product_reordered_mean</th>\n",
       "      <th>product_reordered_std</th>\n",
       "      <th>product_reordered_sum</th>\n",
       "      <th>user_product</th>\n",
       "      <th>user_product_reordered_sum</th>\n",
       "      <th>user_product_add_to_cart_order_sum</th>\n",
       "      <th>user_product_add_to_cart_order_mean</th>\n",
       "      <th>user_product_order_nunique</th>\n",
       "      <th>user_product_last_order_num</th>\n",
       "      <th>product_name</th>\n",
       "      <th>aisle_id</th>\n",
       "      <th>department_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>202279</td>\n",
       "      <td>9327</td>\n",
       "      <td>20227909327</td>\n",
       "      <td>0</td>\n",
       "      <td>1050357</td>\n",
       "      <td>train</td>\n",
       "      <td>9</td>\n",
       "      <td>1</td>\n",
       "      <td>14</td>\n",
       "      <td>30.0</td>\n",
       "      <td>39</td>\n",
       "      <td>9</td>\n",
       "      <td>22.625000</td>\n",
       "      <td>30.0</td>\n",
       "      <td>9.455724</td>\n",
       "      <td>30.0</td>\n",
       "      <td>5</td>\n",
       "      <td>9</td>\n",
       "      <td>134</td>\n",
       "      <td>47</td>\n",
       "      <td>17794</td>\n",
       "      <td>1003</td>\n",
       "      <td>28985</td>\n",
       "      <td>90</td>\n",
       "      <td>10.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>5304</td>\n",
       "      <td>9.943166</td>\n",
       "      <td>7.853449</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>14</td>\n",
       "      <td>13</td>\n",
       "      <td>30.0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>12</td>\n",
       "      <td>13</td>\n",
       "      <td>11.432298</td>\n",
       "      <td>9.184589</td>\n",
       "      <td>0.157962</td>\n",
       "      <td>0.364734</td>\n",
       "      <td>995</td>\n",
       "      <td>20227909327</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Garlic Powder</td>\n",
       "      <td>104</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>156122</td>\n",
       "      <td>47209</td>\n",
       "      <td>15612247209</td>\n",
       "      <td>0</td>\n",
       "      <td>1277092</td>\n",
       "      <td>train</td>\n",
       "      <td>53</td>\n",
       "      <td>1</td>\n",
       "      <td>11</td>\n",
       "      <td>5.0</td>\n",
       "      <td>36</td>\n",
       "      <td>53</td>\n",
       "      <td>6.673077</td>\n",
       "      <td>18.0</td>\n",
       "      <td>2.874643</td>\n",
       "      <td>6.0</td>\n",
       "      <td>4</td>\n",
       "      <td>14</td>\n",
       "      <td>114</td>\n",
       "      <td>154</td>\n",
       "      <td>13176</td>\n",
       "      <td>13176</td>\n",
       "      <td>48370</td>\n",
       "      <td>967</td>\n",
       "      <td>18.245283</td>\n",
       "      <td>...</td>\n",
       "      <td>43453</td>\n",
       "      <td>6.775011</td>\n",
       "      <td>5.813507</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>15</td>\n",
       "      <td>14</td>\n",
       "      <td>7.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>14</td>\n",
       "      <td>15</td>\n",
       "      <td>10.205608</td>\n",
       "      <td>8.319953</td>\n",
       "      <td>0.796553</td>\n",
       "      <td>0.402563</td>\n",
       "      <td>170131</td>\n",
       "      <td>15612247209</td>\n",
       "      <td>0</td>\n",
       "      <td>22</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>42</td>\n",
       "      <td>Organic Hass Avocado</td>\n",
       "      <td>24</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>156122</td>\n",
       "      <td>46522</td>\n",
       "      <td>15612246522</td>\n",
       "      <td>0</td>\n",
       "      <td>1277092</td>\n",
       "      <td>train</td>\n",
       "      <td>53</td>\n",
       "      <td>1</td>\n",
       "      <td>11</td>\n",
       "      <td>5.0</td>\n",
       "      <td>36</td>\n",
       "      <td>53</td>\n",
       "      <td>6.673077</td>\n",
       "      <td>18.0</td>\n",
       "      <td>2.874643</td>\n",
       "      <td>6.0</td>\n",
       "      <td>4</td>\n",
       "      <td>14</td>\n",
       "      <td>114</td>\n",
       "      <td>154</td>\n",
       "      <td>13176</td>\n",
       "      <td>13176</td>\n",
       "      <td>48370</td>\n",
       "      <td>967</td>\n",
       "      <td>18.245283</td>\n",
       "      <td>...</td>\n",
       "      <td>4361</td>\n",
       "      <td>10.115824</td>\n",
       "      <td>7.885046</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>11</td>\n",
       "      <td>10</td>\n",
       "      <td>30.0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>11</td>\n",
       "      <td>12</td>\n",
       "      <td>12.547652</td>\n",
       "      <td>9.160984</td>\n",
       "      <td>0.452754</td>\n",
       "      <td>0.497794</td>\n",
       "      <td>3608</td>\n",
       "      <td>15612246522</td>\n",
       "      <td>0</td>\n",
       "      <td>23</td>\n",
       "      <td>23.0</td>\n",
       "      <td>1</td>\n",
       "      <td>42</td>\n",
       "      <td>Spaghetti Pasta</td>\n",
       "      <td>131</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>156122</td>\n",
       "      <td>38693</td>\n",
       "      <td>15612238693</td>\n",
       "      <td>0</td>\n",
       "      <td>1277092</td>\n",
       "      <td>train</td>\n",
       "      <td>53</td>\n",
       "      <td>1</td>\n",
       "      <td>11</td>\n",
       "      <td>5.0</td>\n",
       "      <td>36</td>\n",
       "      <td>53</td>\n",
       "      <td>6.673077</td>\n",
       "      <td>18.0</td>\n",
       "      <td>2.874643</td>\n",
       "      <td>6.0</td>\n",
       "      <td>4</td>\n",
       "      <td>14</td>\n",
       "      <td>114</td>\n",
       "      <td>154</td>\n",
       "      <td>13176</td>\n",
       "      <td>13176</td>\n",
       "      <td>48370</td>\n",
       "      <td>967</td>\n",
       "      <td>18.245283</td>\n",
       "      <td>...</td>\n",
       "      <td>2546</td>\n",
       "      <td>8.862221</td>\n",
       "      <td>6.974303</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>12</td>\n",
       "      <td>14</td>\n",
       "      <td>7.0</td>\n",
       "      <td>30.0</td>\n",
       "      <td>12</td>\n",
       "      <td>11</td>\n",
       "      <td>10.664021</td>\n",
       "      <td>8.701408</td>\n",
       "      <td>0.437845</td>\n",
       "      <td>0.496176</td>\n",
       "      <td>1983</td>\n",
       "      <td>15612238693</td>\n",
       "      <td>0</td>\n",
       "      <td>24</td>\n",
       "      <td>24.0</td>\n",
       "      <td>1</td>\n",
       "      <td>42</td>\n",
       "      <td>Meyer Lemon</td>\n",
       "      <td>24</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>156122</td>\n",
       "      <td>48825</td>\n",
       "      <td>15612248825</td>\n",
       "      <td>0</td>\n",
       "      <td>1277092</td>\n",
       "      <td>train</td>\n",
       "      <td>53</td>\n",
       "      <td>1</td>\n",
       "      <td>11</td>\n",
       "      <td>5.0</td>\n",
       "      <td>36</td>\n",
       "      <td>53</td>\n",
       "      <td>6.673077</td>\n",
       "      <td>18.0</td>\n",
       "      <td>2.874643</td>\n",
       "      <td>6.0</td>\n",
       "      <td>4</td>\n",
       "      <td>14</td>\n",
       "      <td>114</td>\n",
       "      <td>154</td>\n",
       "      <td>13176</td>\n",
       "      <td>13176</td>\n",
       "      <td>48370</td>\n",
       "      <td>967</td>\n",
       "      <td>18.245283</td>\n",
       "      <td>...</td>\n",
       "      <td>959</td>\n",
       "      <td>9.408513</td>\n",
       "      <td>7.249891</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>12</td>\n",
       "      <td>13</td>\n",
       "      <td>7.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>11</td>\n",
       "      <td>12</td>\n",
       "      <td>9.690413</td>\n",
       "      <td>8.013440</td>\n",
       "      <td>0.390724</td>\n",
       "      <td>0.488068</td>\n",
       "      <td>615</td>\n",
       "      <td>15612248825</td>\n",
       "      <td>0</td>\n",
       "      <td>25</td>\n",
       "      <td>25.0</td>\n",
       "      <td>1</td>\n",
       "      <td>42</td>\n",
       "      <td>Bag of Organic Lemons</td>\n",
       "      <td>24</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 58 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   user_id  product_id  user_X_product  label  order_id eval_set  \\\n",
       "0   202279        9327     20227909327      0   1050357    train   \n",
       "1   156122       47209     15612247209      0   1277092    train   \n",
       "2   156122       46522     15612246522      0   1277092    train   \n",
       "3   156122       38693     15612238693      0   1277092    train   \n",
       "4   156122       48825     15612248825      0   1277092    train   \n",
       "\n",
       "   order_number  order_dow  order_hour_of_day  days_since_prior_order  \\\n",
       "0             9          1                 14                    30.0   \n",
       "1            53          1                 11                     5.0   \n",
       "2            53          1                 11                     5.0   \n",
       "3            53          1                 11                     5.0   \n",
       "4            53          1                 11                     5.0   \n",
       "\n",
       "   dow_hour  user_orderid_count  user_days_since_prior_order_mean  \\\n",
       "0        39                   9                         22.625000   \n",
       "1        36                  53                          6.673077   \n",
       "2        36                  53                          6.673077   \n",
       "3        36                  53                          6.673077   \n",
       "4        36                  53                          6.673077   \n",
       "\n",
       "   user_days_since_prior_order_max  user_days_since_prior_order_std  \\\n",
       "0                             30.0                         9.455724   \n",
       "1                             18.0                         2.874643   \n",
       "2                             18.0                         2.874643   \n",
       "3                             18.0                         2.874643   \n",
       "4                             18.0                         2.874643   \n",
       "\n",
       "   user_days_since_prior_order_mode  user_order_dow_mode  \\\n",
       "0                              30.0                    5   \n",
       "1                               6.0                    4   \n",
       "2                               6.0                    4   \n",
       "3                               6.0                    4   \n",
       "4                               6.0                    4   \n",
       "\n",
       "   user_order_hour_of_day_mode  user_dow_hour_mode  user_product_nunique  \\\n",
       "0                            9                 134                    47   \n",
       "1                           14                 114                   154   \n",
       "2                           14                 114                   154   \n",
       "3                           14                 114                   154   \n",
       "4                           14                 114                   154   \n",
       "\n",
       "   user_product_mode  user_product_mode_1  user_product_mode_2  \\\n",
       "0              17794                 1003                28985   \n",
       "1              13176                13176                48370   \n",
       "2              13176                13176                48370   \n",
       "3              13176                13176                48370   \n",
       "4              13176                13176                48370   \n",
       "\n",
       "   user_product_count  user_product_orderid_ratio      ...        \\\n",
       "0                  90                   10.000000      ...         \n",
       "1                 967                   18.245283      ...         \n",
       "2                 967                   18.245283      ...         \n",
       "3                 967                   18.245283      ...         \n",
       "4                 967                   18.245283      ...         \n",
       "\n",
       "   product_user_nunqiue  product_add_to_cart_order_mean  \\\n",
       "0                  5304                        9.943166   \n",
       "1                 43453                        6.775011   \n",
       "2                  4361                       10.115824   \n",
       "3                  2546                        8.862221   \n",
       "4                   959                        9.408513   \n",
       "\n",
       "   product_add_to_cart_order_std  product_dow_mode  product_dow_mode_1  \\\n",
       "0                       7.853449                 0                   6   \n",
       "1                       5.813507                 0                   1   \n",
       "2                       7.885046                 0                   1   \n",
       "3                       6.974303                 0                   1   \n",
       "4                       7.249891                 0                   6   \n",
       "\n",
       "   product_hour_of_day_mode  product_hour_of_day_mode_1  \\\n",
       "0                        14                          13   \n",
       "1                        15                          14   \n",
       "2                        11                          10   \n",
       "3                        12                          14   \n",
       "4                        12                          13   \n",
       "\n",
       "   product_days_since_prior_order_mode  product_days_since_prior_order_mode_1  \\\n",
       "0                                 30.0                                    7.0   \n",
       "1                                  7.0                                    6.0   \n",
       "2                                 30.0                                    7.0   \n",
       "3                                  7.0                                   30.0   \n",
       "4                                  7.0                                    4.0   \n",
       "\n",
       "   product_dow_hour_mode  product_dow_hour_mode_1  \\\n",
       "0                     12                       13   \n",
       "1                     14                       15   \n",
       "2                     11                       12   \n",
       "3                     12                       11   \n",
       "4                     11                       12   \n",
       "\n",
       "   product_days_since_prior_order_mean  product_days_since_prior_order_std  \\\n",
       "0                            11.432298                            9.184589   \n",
       "1                            10.205608                            8.319953   \n",
       "2                            12.547652                            9.160984   \n",
       "3                            10.664021                            8.701408   \n",
       "4                             9.690413                            8.013440   \n",
       "\n",
       "   product_reordered_mean  product_reordered_std  product_reordered_sum  \\\n",
       "0                0.157962               0.364734                    995   \n",
       "1                0.796553               0.402563                 170131   \n",
       "2                0.452754               0.497794                   3608   \n",
       "3                0.437845               0.496176                   1983   \n",
       "4                0.390724               0.488068                    615   \n",
       "\n",
       "   user_product  user_product_reordered_sum  \\\n",
       "0   20227909327                           0   \n",
       "1   15612247209                           0   \n",
       "2   15612246522                           0   \n",
       "3   15612238693                           0   \n",
       "4   15612248825                           0   \n",
       "\n",
       "   user_product_add_to_cart_order_sum  user_product_add_to_cart_order_mean  \\\n",
       "0                                   3                                  3.0   \n",
       "1                                  22                                 22.0   \n",
       "2                                  23                                 23.0   \n",
       "3                                  24                                 24.0   \n",
       "4                                  25                                 25.0   \n",
       "\n",
       "   user_product_order_nunique  user_product_last_order_num  \\\n",
       "0                           1                            3   \n",
       "1                           1                           42   \n",
       "2                           1                           42   \n",
       "3                           1                           42   \n",
       "4                           1                           42   \n",
       "\n",
       "            product_name  aisle_id  department_id  \n",
       "0          Garlic Powder       104             13  \n",
       "1   Organic Hass Avocado        24              4  \n",
       "2        Spaghetti Pasta       131              9  \n",
       "3            Meyer Lemon        24              4  \n",
       "4  Bag of Organic Lemons        24              4  \n",
       "\n",
       "[5 rows x 58 columns]"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.set_option('display.max_columns',50)\n",
    "train_data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [],
   "source": [
    "test_data = test_data.merge(user_fea, on='user_id', how='left')\n",
    "test_data = test_data.merge(product_fea, on='product_id', how='left')\n",
    "test_data = test_data.merge(userXproduct_fea, left_on='user_X_product', right_on='user_product', how='left')\n",
    "test_data  =  test_data.merge(products, on='product_id', how= 'left')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_id</th>\n",
       "      <th>product_id</th>\n",
       "      <th>user_X_product</th>\n",
       "      <th>label</th>\n",
       "      <th>order_id</th>\n",
       "      <th>eval_set</th>\n",
       "      <th>order_number</th>\n",
       "      <th>order_dow</th>\n",
       "      <th>order_hour_of_day</th>\n",
       "      <th>days_since_prior_order</th>\n",
       "      <th>dow_hour</th>\n",
       "      <th>user_orderid_count</th>\n",
       "      <th>user_days_since_prior_order_mean</th>\n",
       "      <th>user_days_since_prior_order_max</th>\n",
       "      <th>user_days_since_prior_order_std</th>\n",
       "      <th>user_days_since_prior_order_mode</th>\n",
       "      <th>user_order_dow_mode</th>\n",
       "      <th>user_order_hour_of_day_mode</th>\n",
       "      <th>user_dow_hour_mode</th>\n",
       "      <th>user_product_nunique</th>\n",
       "      <th>user_product_mode</th>\n",
       "      <th>user_product_mode_1</th>\n",
       "      <th>user_product_mode_2</th>\n",
       "      <th>user_product_count</th>\n",
       "      <th>user_product_orderid_ratio</th>\n",
       "      <th>...</th>\n",
       "      <th>product_user_nunqiue</th>\n",
       "      <th>product_add_to_cart_order_mean</th>\n",
       "      <th>product_add_to_cart_order_std</th>\n",
       "      <th>product_dow_mode</th>\n",
       "      <th>product_dow_mode_1</th>\n",
       "      <th>product_hour_of_day_mode</th>\n",
       "      <th>product_hour_of_day_mode_1</th>\n",
       "      <th>product_days_since_prior_order_mode</th>\n",
       "      <th>product_days_since_prior_order_mode_1</th>\n",
       "      <th>product_dow_hour_mode</th>\n",
       "      <th>product_dow_hour_mode_1</th>\n",
       "      <th>product_days_since_prior_order_mean</th>\n",
       "      <th>product_days_since_prior_order_std</th>\n",
       "      <th>product_reordered_mean</th>\n",
       "      <th>product_reordered_std</th>\n",
       "      <th>product_reordered_sum</th>\n",
       "      <th>user_product</th>\n",
       "      <th>user_product_reordered_sum</th>\n",
       "      <th>user_product_add_to_cart_order_sum</th>\n",
       "      <th>user_product_add_to_cart_order_mean</th>\n",
       "      <th>user_product_order_nunique</th>\n",
       "      <th>user_product_last_order_num</th>\n",
       "      <th>product_name</th>\n",
       "      <th>aisle_id</th>\n",
       "      <th>department_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>202279</td>\n",
       "      <td>9327</td>\n",
       "      <td>20227909327</td>\n",
       "      <td>0</td>\n",
       "      <td>1050357</td>\n",
       "      <td>train</td>\n",
       "      <td>9</td>\n",
       "      <td>1</td>\n",
       "      <td>14</td>\n",
       "      <td>30.0</td>\n",
       "      <td>39</td>\n",
       "      <td>9</td>\n",
       "      <td>22.625000</td>\n",
       "      <td>30.0</td>\n",
       "      <td>9.455724</td>\n",
       "      <td>30.0</td>\n",
       "      <td>5</td>\n",
       "      <td>9</td>\n",
       "      <td>134</td>\n",
       "      <td>47</td>\n",
       "      <td>17794</td>\n",
       "      <td>1003</td>\n",
       "      <td>28985</td>\n",
       "      <td>90</td>\n",
       "      <td>10.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>5304</td>\n",
       "      <td>9.943166</td>\n",
       "      <td>7.853449</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>14</td>\n",
       "      <td>13</td>\n",
       "      <td>30.0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>12</td>\n",
       "      <td>13</td>\n",
       "      <td>11.432298</td>\n",
       "      <td>9.184589</td>\n",
       "      <td>0.157962</td>\n",
       "      <td>0.364734</td>\n",
       "      <td>995</td>\n",
       "      <td>20227909327</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Garlic Powder</td>\n",
       "      <td>104</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>156122</td>\n",
       "      <td>47209</td>\n",
       "      <td>15612247209</td>\n",
       "      <td>0</td>\n",
       "      <td>1277092</td>\n",
       "      <td>train</td>\n",
       "      <td>53</td>\n",
       "      <td>1</td>\n",
       "      <td>11</td>\n",
       "      <td>5.0</td>\n",
       "      <td>36</td>\n",
       "      <td>53</td>\n",
       "      <td>6.673077</td>\n",
       "      <td>18.0</td>\n",
       "      <td>2.874643</td>\n",
       "      <td>6.0</td>\n",
       "      <td>4</td>\n",
       "      <td>14</td>\n",
       "      <td>114</td>\n",
       "      <td>154</td>\n",
       "      <td>13176</td>\n",
       "      <td>13176</td>\n",
       "      <td>48370</td>\n",
       "      <td>967</td>\n",
       "      <td>18.245283</td>\n",
       "      <td>...</td>\n",
       "      <td>43453</td>\n",
       "      <td>6.775011</td>\n",
       "      <td>5.813507</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>15</td>\n",
       "      <td>14</td>\n",
       "      <td>7.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>14</td>\n",
       "      <td>15</td>\n",
       "      <td>10.205608</td>\n",
       "      <td>8.319953</td>\n",
       "      <td>0.796553</td>\n",
       "      <td>0.402563</td>\n",
       "      <td>170131</td>\n",
       "      <td>15612247209</td>\n",
       "      <td>0</td>\n",
       "      <td>22</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>42</td>\n",
       "      <td>Organic Hass Avocado</td>\n",
       "      <td>24</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>156122</td>\n",
       "      <td>46522</td>\n",
       "      <td>15612246522</td>\n",
       "      <td>0</td>\n",
       "      <td>1277092</td>\n",
       "      <td>train</td>\n",
       "      <td>53</td>\n",
       "      <td>1</td>\n",
       "      <td>11</td>\n",
       "      <td>5.0</td>\n",
       "      <td>36</td>\n",
       "      <td>53</td>\n",
       "      <td>6.673077</td>\n",
       "      <td>18.0</td>\n",
       "      <td>2.874643</td>\n",
       "      <td>6.0</td>\n",
       "      <td>4</td>\n",
       "      <td>14</td>\n",
       "      <td>114</td>\n",
       "      <td>154</td>\n",
       "      <td>13176</td>\n",
       "      <td>13176</td>\n",
       "      <td>48370</td>\n",
       "      <td>967</td>\n",
       "      <td>18.245283</td>\n",
       "      <td>...</td>\n",
       "      <td>4361</td>\n",
       "      <td>10.115824</td>\n",
       "      <td>7.885046</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>11</td>\n",
       "      <td>10</td>\n",
       "      <td>30.0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>11</td>\n",
       "      <td>12</td>\n",
       "      <td>12.547652</td>\n",
       "      <td>9.160984</td>\n",
       "      <td>0.452754</td>\n",
       "      <td>0.497794</td>\n",
       "      <td>3608</td>\n",
       "      <td>15612246522</td>\n",
       "      <td>0</td>\n",
       "      <td>23</td>\n",
       "      <td>23.0</td>\n",
       "      <td>1</td>\n",
       "      <td>42</td>\n",
       "      <td>Spaghetti Pasta</td>\n",
       "      <td>131</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>156122</td>\n",
       "      <td>38693</td>\n",
       "      <td>15612238693</td>\n",
       "      <td>0</td>\n",
       "      <td>1277092</td>\n",
       "      <td>train</td>\n",
       "      <td>53</td>\n",
       "      <td>1</td>\n",
       "      <td>11</td>\n",
       "      <td>5.0</td>\n",
       "      <td>36</td>\n",
       "      <td>53</td>\n",
       "      <td>6.673077</td>\n",
       "      <td>18.0</td>\n",
       "      <td>2.874643</td>\n",
       "      <td>6.0</td>\n",
       "      <td>4</td>\n",
       "      <td>14</td>\n",
       "      <td>114</td>\n",
       "      <td>154</td>\n",
       "      <td>13176</td>\n",
       "      <td>13176</td>\n",
       "      <td>48370</td>\n",
       "      <td>967</td>\n",
       "      <td>18.245283</td>\n",
       "      <td>...</td>\n",
       "      <td>2546</td>\n",
       "      <td>8.862221</td>\n",
       "      <td>6.974303</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>12</td>\n",
       "      <td>14</td>\n",
       "      <td>7.0</td>\n",
       "      <td>30.0</td>\n",
       "      <td>12</td>\n",
       "      <td>11</td>\n",
       "      <td>10.664021</td>\n",
       "      <td>8.701408</td>\n",
       "      <td>0.437845</td>\n",
       "      <td>0.496176</td>\n",
       "      <td>1983</td>\n",
       "      <td>15612238693</td>\n",
       "      <td>0</td>\n",
       "      <td>24</td>\n",
       "      <td>24.0</td>\n",
       "      <td>1</td>\n",
       "      <td>42</td>\n",
       "      <td>Meyer Lemon</td>\n",
       "      <td>24</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>156122</td>\n",
       "      <td>48825</td>\n",
       "      <td>15612248825</td>\n",
       "      <td>0</td>\n",
       "      <td>1277092</td>\n",
       "      <td>train</td>\n",
       "      <td>53</td>\n",
       "      <td>1</td>\n",
       "      <td>11</td>\n",
       "      <td>5.0</td>\n",
       "      <td>36</td>\n",
       "      <td>53</td>\n",
       "      <td>6.673077</td>\n",
       "      <td>18.0</td>\n",
       "      <td>2.874643</td>\n",
       "      <td>6.0</td>\n",
       "      <td>4</td>\n",
       "      <td>14</td>\n",
       "      <td>114</td>\n",
       "      <td>154</td>\n",
       "      <td>13176</td>\n",
       "      <td>13176</td>\n",
       "      <td>48370</td>\n",
       "      <td>967</td>\n",
       "      <td>18.245283</td>\n",
       "      <td>...</td>\n",
       "      <td>959</td>\n",
       "      <td>9.408513</td>\n",
       "      <td>7.249891</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>12</td>\n",
       "      <td>13</td>\n",
       "      <td>7.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>11</td>\n",
       "      <td>12</td>\n",
       "      <td>9.690413</td>\n",
       "      <td>8.013440</td>\n",
       "      <td>0.390724</td>\n",
       "      <td>0.488068</td>\n",
       "      <td>615</td>\n",
       "      <td>15612248825</td>\n",
       "      <td>0</td>\n",
       "      <td>25</td>\n",
       "      <td>25.0</td>\n",
       "      <td>1</td>\n",
       "      <td>42</td>\n",
       "      <td>Bag of Organic Lemons</td>\n",
       "      <td>24</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 58 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   user_id  product_id  user_X_product  label  order_id eval_set  \\\n",
       "0   202279        9327     20227909327      0   1050357    train   \n",
       "1   156122       47209     15612247209      0   1277092    train   \n",
       "2   156122       46522     15612246522      0   1277092    train   \n",
       "3   156122       38693     15612238693      0   1277092    train   \n",
       "4   156122       48825     15612248825      0   1277092    train   \n",
       "\n",
       "   order_number  order_dow  order_hour_of_day  days_since_prior_order  \\\n",
       "0             9          1                 14                    30.0   \n",
       "1            53          1                 11                     5.0   \n",
       "2            53          1                 11                     5.0   \n",
       "3            53          1                 11                     5.0   \n",
       "4            53          1                 11                     5.0   \n",
       "\n",
       "   dow_hour  user_orderid_count  user_days_since_prior_order_mean  \\\n",
       "0        39                   9                         22.625000   \n",
       "1        36                  53                          6.673077   \n",
       "2        36                  53                          6.673077   \n",
       "3        36                  53                          6.673077   \n",
       "4        36                  53                          6.673077   \n",
       "\n",
       "   user_days_since_prior_order_max  user_days_since_prior_order_std  \\\n",
       "0                             30.0                         9.455724   \n",
       "1                             18.0                         2.874643   \n",
       "2                             18.0                         2.874643   \n",
       "3                             18.0                         2.874643   \n",
       "4                             18.0                         2.874643   \n",
       "\n",
       "   user_days_since_prior_order_mode  user_order_dow_mode  \\\n",
       "0                              30.0                    5   \n",
       "1                               6.0                    4   \n",
       "2                               6.0                    4   \n",
       "3                               6.0                    4   \n",
       "4                               6.0                    4   \n",
       "\n",
       "   user_order_hour_of_day_mode  user_dow_hour_mode  user_product_nunique  \\\n",
       "0                            9                 134                    47   \n",
       "1                           14                 114                   154   \n",
       "2                           14                 114                   154   \n",
       "3                           14                 114                   154   \n",
       "4                           14                 114                   154   \n",
       "\n",
       "   user_product_mode  user_product_mode_1  user_product_mode_2  \\\n",
       "0              17794                 1003                28985   \n",
       "1              13176                13176                48370   \n",
       "2              13176                13176                48370   \n",
       "3              13176                13176                48370   \n",
       "4              13176                13176                48370   \n",
       "\n",
       "   user_product_count  user_product_orderid_ratio      ...        \\\n",
       "0                  90                   10.000000      ...         \n",
       "1                 967                   18.245283      ...         \n",
       "2                 967                   18.245283      ...         \n",
       "3                 967                   18.245283      ...         \n",
       "4                 967                   18.245283      ...         \n",
       "\n",
       "   product_user_nunqiue  product_add_to_cart_order_mean  \\\n",
       "0                  5304                        9.943166   \n",
       "1                 43453                        6.775011   \n",
       "2                  4361                       10.115824   \n",
       "3                  2546                        8.862221   \n",
       "4                   959                        9.408513   \n",
       "\n",
       "   product_add_to_cart_order_std  product_dow_mode  product_dow_mode_1  \\\n",
       "0                       7.853449                 0                   6   \n",
       "1                       5.813507                 0                   1   \n",
       "2                       7.885046                 0                   1   \n",
       "3                       6.974303                 0                   1   \n",
       "4                       7.249891                 0                   6   \n",
       "\n",
       "   product_hour_of_day_mode  product_hour_of_day_mode_1  \\\n",
       "0                        14                          13   \n",
       "1                        15                          14   \n",
       "2                        11                          10   \n",
       "3                        12                          14   \n",
       "4                        12                          13   \n",
       "\n",
       "   product_days_since_prior_order_mode  product_days_since_prior_order_mode_1  \\\n",
       "0                                 30.0                                    7.0   \n",
       "1                                  7.0                                    6.0   \n",
       "2                                 30.0                                    7.0   \n",
       "3                                  7.0                                   30.0   \n",
       "4                                  7.0                                    4.0   \n",
       "\n",
       "   product_dow_hour_mode  product_dow_hour_mode_1  \\\n",
       "0                     12                       13   \n",
       "1                     14                       15   \n",
       "2                     11                       12   \n",
       "3                     12                       11   \n",
       "4                     11                       12   \n",
       "\n",
       "   product_days_since_prior_order_mean  product_days_since_prior_order_std  \\\n",
       "0                            11.432298                            9.184589   \n",
       "1                            10.205608                            8.319953   \n",
       "2                            12.547652                            9.160984   \n",
       "3                            10.664021                            8.701408   \n",
       "4                             9.690413                            8.013440   \n",
       "\n",
       "   product_reordered_mean  product_reordered_std  product_reordered_sum  \\\n",
       "0                0.157962               0.364734                    995   \n",
       "1                0.796553               0.402563                 170131   \n",
       "2                0.452754               0.497794                   3608   \n",
       "3                0.437845               0.496176                   1983   \n",
       "4                0.390724               0.488068                    615   \n",
       "\n",
       "   user_product  user_product_reordered_sum  \\\n",
       "0   20227909327                           0   \n",
       "1   15612247209                           0   \n",
       "2   15612246522                           0   \n",
       "3   15612238693                           0   \n",
       "4   15612248825                           0   \n",
       "\n",
       "   user_product_add_to_cart_order_sum  user_product_add_to_cart_order_mean  \\\n",
       "0                                   3                                  3.0   \n",
       "1                                  22                                 22.0   \n",
       "2                                  23                                 23.0   \n",
       "3                                  24                                 24.0   \n",
       "4                                  25                                 25.0   \n",
       "\n",
       "   user_product_order_nunique  user_product_last_order_num  \\\n",
       "0                           1                            3   \n",
       "1                           1                           42   \n",
       "2                           1                           42   \n",
       "3                           1                           42   \n",
       "4                           1                           42   \n",
       "\n",
       "            product_name  aisle_id  department_id  \n",
       "0          Garlic Powder       104             13  \n",
       "1   Organic Hass Avocado        24              4  \n",
       "2        Spaghetti Pasta       131              9  \n",
       "3            Meyer Lemon        24              4  \n",
       "4  Bag of Organic Lemons        24              4  \n",
       "\n",
       "[5 rows x 58 columns]"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "train_data['user_product_count_percent']  = train_data['user_product_order_nunique'] * 1.0 / train_data['user_orderid_count']\n",
    "train_data['user_product_last_to_now']   = train_data['user_orderid_count']    - train_data['user_product_last_order_num']\n",
    "\n",
    "test_data['user_product_count_percent']   = test_data['user_product_order_nunique'] * 1.0  / test_data['user_orderid_count']\n",
    "test_data['user_product_last_to_now']   = test_data['user_orderid_count']    - test_data['user_product_last_order_num']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [],
   "source": [
    "fea_not_need = ['user_id','user_X_product','order_id','eval_set','product_name','user_product_last_order_num','label']\n",
    "feature_cols = [col for col in train_data.columns if col not in fea_not_need]\n",
    "label_cols = 'label'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [],
   "source": [
    "def validation_sample(order_ids, frac = 0.2):\n",
    "    import random\n",
    "    sample_number = int(frac * len(order_ids))\n",
    "    sample_val_order = random.sample( order_ids , sample_number) \n",
    "    sample_train_order = list(set(order_ids) - set(sample_val_order))\n",
    "    return sample_train_order,sample_val_order\n",
    "sample_train_order,sample_val_order = validation_sample(list(train_data['order_id'].unique()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [],
   "source": [
    "train = train_data.loc[train_data.order_id.isin(sample_train_order)]\n",
    "val   = train_data.loc[train_data.order_id.isin(sample_val_order)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 模型验证\n",
    "### 近似的评价指标"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [],
   "source": [
    "import lightgbm as lgb\n",
    "\n",
    "from sklearn.metrics import f1_score\n",
    "def lgb_f1_score(y_hat, data):\n",
    "    y_true = data.get_label() \n",
    "    y_hat = np.round(y_hat >= 0.2) # scikits f1 doesn't like probabilities \n",
    "    return 'f1', f1_score(y_true, y_hat), True"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 模型验证 \n",
    "此处是近似的验证,我们仅仅作为参考(未加入None等),在测试集上应该会有波动(我们的训练集&测试集的比例和线上不是一致的,为了方便我们此处随机采样了20%)\n",
    "\n",
    "<font color=red>注意: 此处我们的F1并不是Mean F1 Score,具体的Mean F1 Score的函数我们会在下一次给出,所以这边的线下的F1 Score和线上的Mean F1 Score基本是不符的。 </font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "light GBM train :-)\n",
      "[10]\ttraining's binary_logloss: 0.247852\ttraining's f1: 0.432576\tvalid_1's binary_logloss: 0.249092\tvalid_1's f1: 0.432067\n",
      "[20]\ttraining's binary_logloss: 0.246291\ttraining's f1: 0.436238\tvalid_1's binary_logloss: 0.247785\tvalid_1's f1: 0.43522\n",
      "[30]\ttraining's binary_logloss: 0.245689\ttraining's f1: 0.437799\tvalid_1's binary_logloss: 0.247615\tvalid_1's f1: 0.436006\n",
      "[40]\ttraining's binary_logloss: 0.24537\ttraining's f1: 0.438867\tvalid_1's binary_logloss: 0.248081\tvalid_1's f1: 0.435775\n",
      "[50]\ttraining's binary_logloss: 0.24519\ttraining's f1: 0.439777\tvalid_1's binary_logloss: 0.248307\tvalid_1's f1: 0.435959\n",
      "[60]\ttraining's binary_logloss: 0.245059\ttraining's f1: 0.440385\tvalid_1's binary_logloss: 0.248371\tvalid_1's f1: 0.435946\n",
      "[70]\ttraining's binary_logloss: 0.245853\ttraining's f1: 0.440425\tvalid_1's binary_logloss: 0.252598\tvalid_1's f1: 0.435574\n",
      "[80]\ttraining's binary_logloss: 0.245137\ttraining's f1: 0.440693\tvalid_1's binary_logloss: 0.251947\tvalid_1's f1: 0.435473\n",
      "[90]\ttraining's binary_logloss: 0.24568\ttraining's f1: 0.440836\tvalid_1's binary_logloss: 0.252623\tvalid_1's f1: 0.435346\n",
      "[100]\ttraining's binary_logloss: 0.246542\ttraining's f1: 0.441033\tvalid_1's binary_logloss: 0.253208\tvalid_1's f1: 0.435358\n",
      "[110]\ttraining's binary_logloss: 0.245438\ttraining's f1: 0.441253\tvalid_1's binary_logloss: 0.252414\tvalid_1's f1: 0.435297\n",
      "[120]\ttraining's binary_logloss: 0.246559\ttraining's f1: 0.44134\tvalid_1's binary_logloss: 0.254097\tvalid_1's f1: 0.435127\n",
      "[130]\ttraining's binary_logloss: 0.245415\ttraining's f1: 0.441625\tvalid_1's binary_logloss: 0.253504\tvalid_1's f1: 0.435102\n",
      "[140]\ttraining's binary_logloss: 0.245632\ttraining's f1: 0.441667\tvalid_1's binary_logloss: 0.253478\tvalid_1's f1: 0.435105\n",
      "[150]\ttraining's binary_logloss: 0.246766\ttraining's f1: 0.441664\tvalid_1's binary_logloss: 0.254114\tvalid_1's f1: 0.434991\n",
      "[160]\ttraining's binary_logloss: 0.246914\ttraining's f1: 0.441765\tvalid_1's binary_logloss: 0.254341\tvalid_1's f1: 0.434976\n",
      "[170]\ttraining's binary_logloss: 0.246404\ttraining's f1: 0.44176\tvalid_1's binary_logloss: 0.253766\tvalid_1's f1: 0.435034\n",
      "[180]\ttraining's binary_logloss: 0.246021\ttraining's f1: 0.441937\tvalid_1's binary_logloss: 0.253914\tvalid_1's f1: 0.435268\n",
      "[190]\ttraining's binary_logloss: 0.246367\ttraining's f1: 0.442058\tvalid_1's binary_logloss: 0.255145\tvalid_1's f1: 0.435114\n",
      "[200]\ttraining's binary_logloss: 0.246671\ttraining's f1: 0.442167\tvalid_1's binary_logloss: 0.255371\tvalid_1's f1: 0.43497\n"
     ]
    }
   ],
   "source": [
    "d_train = lgb.Dataset(train[feature_cols], label=train[label_cols].values)   \n",
    "d_val   = lgb.Dataset(val[feature_cols], label=val[label_cols].values)    \n",
    "\n",
    "params = {\n",
    "    'task': 'train',\n",
    "    'boosting_type': 'gbdt',\n",
    "    'objective': 'binary',\n",
    "    'metric': {'binary_logloss'},\n",
    "    'num_leaves': 2 ** 5,\n",
    "    'max_depth': 10,\n",
    "    'learning_rate':0.5,\n",
    "    'feature_fraction': 0.9,\n",
    "    'bagging_fraction': 0.9,\n",
    "    'bagging_freq': 5\n",
    "}\n",
    "ROUNDS = 200 \n",
    "print('light GBM train :-)')\n",
    "bst = lgb.train(params, d_train, ROUNDS, valid_sets=[d_train,d_val], feval =lgb_f1_score,verbose_eval=10)\n",
    "del d_train "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 模型训练&提交\n",
    "### 模型训练"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "light GBM train :-)\n",
      "[10]\ttraining's binary_logloss: 0.257502\ttraining's f1: 0.433764\n",
      "[20]\ttraining's binary_logloss: 0.248057\ttraining's f1: 0.439856\n",
      "[30]\ttraining's binary_logloss: 0.245456\ttraining's f1: 0.440926\n",
      "[40]\ttraining's binary_logloss: 0.244276\ttraining's f1: 0.442424\n",
      "[50]\ttraining's binary_logloss: 0.243474\ttraining's f1: 0.443839\n",
      "[60]\ttraining's binary_logloss: 0.242821\ttraining's f1: 0.445005\n",
      "[70]\ttraining's binary_logloss: 0.242281\ttraining's f1: 0.446259\n",
      "[80]\ttraining's binary_logloss: 0.241743\ttraining's f1: 0.447373\n",
      "[90]\ttraining's binary_logloss: 0.241224\ttraining's f1: 0.448543\n",
      "[100]\ttraining's binary_logloss: 0.240748\ttraining's f1: 0.449529\n",
      "[110]\ttraining's binary_logloss: 0.24033\ttraining's f1: 0.450389\n",
      "[120]\ttraining's binary_logloss: 0.239913\ttraining's f1: 0.451297\n"
     ]
    }
   ],
   "source": [
    "import lightgbm as lgb\n",
    "d_train = lgb.Dataset(train_data[feature_cols], label=train_data[label_cols].values)    \n",
    "\n",
    "params = {\n",
    "    'task': 'train',\n",
    "    'boosting_type': 'gbdt',\n",
    "    'objective': 'binary',\n",
    "    'metric': {'binary_logloss'},\n",
    "    'num_leaves': 255,\n",
    "    'max_depth': 10,\n",
    "#     'feature_fraction': 0.9,\n",
    "#     'bagging_fraction': 0.95,\n",
    "    'bagging_freq': 5\n",
    "}\n",
    "ROUNDS = 120 \n",
    "print('light GBM train :-)')\n",
    "bst = lgb.train(params, d_train, ROUNDS, valid_sets=[d_train], feval =lgb_f1_score,verbose_eval=10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " ### 模型提交:0.3787363（1411/2623）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [],
   "source": [
    "pred = bst.predict(test_data[feature_cols])\n",
    "test_data['pred'] = pred \n",
    "test_data['product_id'] = test_data['product_id'].astype(str)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [],
   "source": [
    "order_product = {}\n",
    "for order_id, val, product_id in test_data[['order_id','pred','product_id']].values:\n",
    "    if order_id in order_product.keys():\n",
    "        if val >= 0.21:\n",
    "            if order_product[order_id] == '':\n",
    "                order_product[order_id] = str(product_id )\n",
    "            else:\n",
    "                order_product[order_id] += ' ' + str(product_id )\n",
    "    else:\n",
    "        order_product[order_id] = ''\n",
    "        if val >= 0.21:\n",
    "            order_product[order_id] = str(product_id ) \n",
    "\n",
    "sub = pd.DataFrame.from_dict(order_product, orient='index')\n",
    "sub.reset_index(inplace=True)\n",
    "sub.columns = ['order_id', 'products']\n",
    "sub.loc[sub.products =='', 'products'] = 'None'\n",
    "sub.to_csv('simple_fe_0.21.csv',index = None)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  },
  "toc": {
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {
    "height": "calc(100% - 180px)",
    "left": "10px",
    "top": "150px",
    "width": "384px"
   },
   "toc_section_display": true,
   "toc_window_display": true
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
